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Part 2: Pre-Service and In-Service Teacher Training & Learning and Teaching Styles 


MILO MILEFF 

INSTRUCTIONAL OBJECTIVES: SELECTING AND DEVISING TASKS 

Introduction 

In the present paper and the discussion that follows, I present aspects of test 
construction and a careful description of instructional objectives. Constructing tests 
involves several stages such as describing language objectives, selecting appropriate 
test task, devising and assembling test tasks, and devising a scoring system for 
evaluation purposes. Tests can be classified according to their reference, that is, 
something external that serves to give test scores meaning. Thus, I distinguish 
between norm referenced and domain referenced testing. The former are in reference 
to performance and the latter to a general domain of skills and knowledge. In other 
instances, tests can derive significance through reference to more restricted domains 
such as those delineated in instructional plans and practices. These are called 
objective referenced tests. The focus here is on objective referenced tests because of 
their particular relevance to evaluation in second language classroom. I also present 
guidelines for devising closed-ended and open-ended test tasks. These guidelines are 
part of a larger process of devising valid tests that are compatible with the focus, 
range, and standards of performance specified or included in instructional 
objectives. 

Instructional objectives 

I distinguish between two sorts of objectives based on their generality. There are 
general instructional objectives, which include the skills, abilities, or knowledge 
students are expected to learn from extended instruction, such as an entire course. 
They are often expressed as the overarching goals of the course. In addition, there 
are specific instructional objectives that are reflected in day-to-day instruction of 
units and lessons. These more specific kinds of objectives are variously referred to 
as syllabus, unit, or lesson objectives and they are associated with course objectives 
around which actual instruction is usually planned and built. Although there are 
different kinds of objectives I focus here on language learning objectives. It is 
crucial that the description of language objectives be accurate and complete and in a 
form that is useful for making tests. I point out two methods of describing language 
objectives for the purposes of devising objectives-referenced tests. 

Objectives as tasks 

The most important way of stating language objectives is in terms of test tasks. 
Consider three examples which by no means exhaust all possible objectives a 
teacher might expect to find. The first example has a skill focus. Here, students will 
be able to answer factual questions including in their answers levels of probability. 
The second example has a structure focus in which students should be able to 
answer correctly multiple-choice items related to vocabulary, syntactic structure, 
and morphological structure. The third example entails communicative or notional 
focus, where students should have the ability to sustain a conversation in English or 
any L2 for that matter about every day affairs and popular topics. Objectives 
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described in this way provide not only the form of the tasks, namely, conversation, 
multiple-choice, etc., but also the standard of performance expected in terms of time 
framing, percentage or degree of success, etc. The second advantage has to do with 
the validity of the test, which represents the skills one wants to measure. Yet another 
advantage to this method is that it allows teachers and students to focus their efforts 
due to a clear and precise statement of objectives. However, some drawbacks do 
exist. In utilizing this method there can be a tendency to concentrate on skills that 
are easy to test while leaving out other skills not so easy to test. In other words, 
objectives may be selected because of their amenability to testing rather than their 
value to language learners. Therefore, if objectives are described independently, this 
problem can be overcome. 

Objectives for learning 

Here, I consider the linguistic content focus and content range of the objective 
in addition to the standard of performance expected of the students. 

Content focus 

Instructional objectives can be described in terms of performance skills (such as 
reading or speaking), communicative language use (including reference to specific 
notions L2 learners will be able to express), specific structures (such as word order 
or pronominal reference), or a combination of these. In any case, the linguistic 
content that is the focus of instruction should be clearly identified in the objective. 
This is necessary when appropriate tasks for the measurement of these skills are 
selected for use in tests. In other words the linguistic content described in the 
language objectives determines the linguistic focus of the test tasks if they are to be 
a valid indicator of attainment of the objectives. 

Content range 

The range of objectives refers not only to the nature of the linguistic or content 
to be learned but also to the conditions in which learners are expected to 
demonstrate their skills. The range can be specified in several ways. One is by 
identifying the issues, topics, or themes that the learners will be able to handle 
utilizing their new language skills. For instance, if the linguistic focus in the 
objective is syntax, one aspect of the range might be that learners are expected to be 
able to parse out sentences with comprehension. This is referred to as thematic range 
of objective. Another range specified in the objective might be the style or genre of 
language that learners are expected to cope with such as formal versus informal 
language. This is referred to as the stylistic range of the objective. A third aspect of 
range concerns the functional range of the skills to be acquired. 

However, instructional objectives often specify one aspect of language content 
and leave other features unspecified, which may result in such objectives being 
overly general. That is, if the objective is expressed in terms of linguistic structures 
then the range must be specified in terms of performance skills and even 
communication skills. As an example, let us say that the instantiation of syntactic A 
movement versus A’ movement is identified as the instructional objective for a 
lesson. This represents a structural skill. The objective must be expanded to identify 
implications of the instantiation in oral or written language and/or comprehension. 
Moreover, are the students expected to formulate hypotheses or conjecture about 
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other syntactic rules? These aspects of the objective need to be specified in order to 
narrow down the initial structural objective. 

To summarize, in order for language objectives to be useful in test development, 
it is important to specify the range of language learning that is expected be it 
thematic, stylistic, and/or functional. 

Standards of performance 

Standards basically tell us how to determine whether the objective has been 
attained. They specify both the quality and the level of performance that students are 
expected to acquire if L2 learning is to be successful. Two of the most common 
bases for assessing linguistic proficiency are accuracy and effectiveness. Accuracy 
can be used as the basis for assessing spelling and grammar in written language 
whereas effectiveness is often used for assessing communication skills. Another 
frequently assessed quality is appropriateness such as familiarity with the use of 
formal and informal modes of communication in spoken language. Other qualities 
used in assessment are authenticity and quantity. They are used mostly when 
evaluating speaking and writing skills and they reflect, for example, how long a 
student can speak on a topic or how close a learner is to what a native speaker might 
say or do. Speed is yet another quality that is sometimes assessed with respect to 
reading, writing, or speaking. While quality tells the instructor what aspects of 
linguistic performance to consider when assessing proficiency on a topic, level is a 
matter of how good a student performance must be with respect to a particular 
quality so that the teacher may conclude that the objective has been attained. A 
useful way of defining levels of performance is to provide models or samples of 
performance that reflect the level of performance one expects of successful L2 
learners. Otherwise this way of describing language objectives may prove to be 
potentially limiting because language skills that cannot be tested easily may be 
excluded. 

Selecting and devising tasks 

Once language learning objectives have been identified and defined, the next 
task is to select those we wish to test. It is preferable to choose a sample of 
objectives to test before beginning test construction in order to avoid biases that can 
arise otherwise such as items that are easier to test or topics that stood out in class 
because they were recently taught. Two commonly used procedures for selecting 
objectives to test are random sampling and stratified sampling procedures. Random 
sampling of objectives is recommended only if all objectives are equally important. 
Otherwise teachers run the risk of not including important objectives and therefore 
not testing them. Stratified sampling on the other hand requires that all possible 
objectives be identified and then organized according to some criterion of 
importance, namely communicative, structural, and/or functional. 

Clearly, the most important factor to consider when choosing which type of test 
task to use is the overall objective. Closed-ended tasks allow assessment of 
comprehension skills in both reading and listening but not speaking or writing. That 
is, a learner’s ability to perform on a closed-ended task does not necessarily mean 
that the individual would be able to produce the corresponding language in an open- 
ended task. Also related to language objectives, closed-ended tasks permit the 
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examiner to assess specific language skills since they are controlled totally by the 
examiner. In comparison, open-ended response tasks do not control the students’ 
specific responses and they can often find ways of responding to test items that are 
different from what was intended by the examiner. 

At this point, I wish to point out that it is important to realize that not all 
authentic language tasks are open-ended. For example, filling out forms are quite 
formulaic. In addition, not all authentic language use involves oral communication. 
Reading and writing are also characteristic of authentic language use. Even multiple- 
choice tests can be an authentic language task for L2 learners in schools where the 
medium of instruction is L2. Because they are less structured than closed-ended 
tasks, open-ended tasks are often used to assess the skills of advanced learners. In 
contrast, low level learners do need the structure imposed by closed-ended test tasks. 
However, multiple-choice tasks for testing beginning level learners can be 
demanding if care is not taken to avoid unnecessary complications. Open-ended 
tasks are suitable for testing speaking and writing skills because they require 
language production. In fact, they call for a variety of language skills such as 
spelling, vocabulary, and grammar skills in addition to discourse and sociolinguistic 
skills. 

Scoring 

Because the specific responses to be made by test-takers in open-ended tasks are 
not controlled in any precise way, devising such tasks does not require the same 
technical precision as closed-ended tasks. Open-ended tests are different from 
closed-ended tests in that they usually consist of only one item (essay), although this 
is not always the case. In contrast, tests made up of closed-ended tasks generally 
include a number of items. 

Domain reference 

Test scores can be interpreted with reference to domains of skills or knowledge. 
Use of domains for test construction purpose requires consensus on what knowledge 
or skills comprise the discipline of interest. The important point here is that in order 
to be a frame of reference for test construction and interpretation, a domain must be 
finite and known. If this is done, it then becomes possible to assess performance on 
the test with reference to how much of the domain in question has been mastered by 
individual L2 learners. Thus, whereas norm-referenced tests provide interpretations 
of test scores relative to other learners, domain-referenced tests provide 
interpretations of test scores relative to an identified domain of knowledge or skill. 

Objectives reference 

Objective referencing is similar to domain referencing in that it provides for the 
interpretations of test scores with respect to a defined area of knowledge or skills. 
The main difference is that it does not require consensus on the description of a 
domain or a field of study. Instead, it depends on the description of the knowledge 
or skills that make up a particular lesson or course. What is important about 
objective referencing is that the domain is conceptualized in local instructional 
terms. This means that, generally speaking, students should be tested in ways that 
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resemble how they were taught (only if the instructional methods are an appropriate 
reflection of the instructional objectives). 

Final word 

When selecting appropriate test tasks, one should take into account the 
instructional activities that have been used in class. Students may not be able to 
demonstrate the full extent of their proficiency if a test task is selected that they have 
not seen before because the task demands may not be clear to them. Surely, an 
indication of language proficiency is the ability to use language in different 
situations. Judgment is called for when choosing test tasks that are different from 
but related to the activities used in class. In some cases, special efforts need to be 
taken to simulate the performance demands of authentic situations in which L2 will 
ultimately be used because the target situations are different from those in the 
classroom. 
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